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DETAILED ACTION 



Response to Arguments 



Applicant's arguments with respect to claims 1-29 have been considered but are moot in 



view of the new ground(s) of rejection. 



Claim Rejections - 35 USC § 102 



2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

3. Claims 1-11, 16-24, 28-29 rejected under 35 U.S.C. 102(e) as being anticipated by 
Schofield et al. (US 6.498,620 B2). 

Re claim 1, Schofield discloses an image processing apparatus comprising: 
an image processing part (18 of fig. 5) for receiving images captured by a plurality of 
cameras (14 and 16 of fig. 7) shooting surroundings of a vehicle (12 of fig. 1) to generate a 
synthetic image (col. 5, lines 59-64) viewed from a virtual point of view (col. 6, lines 1-4) from 
these camera images (col 5, line 65-col. 6, line 4), 

wherein the image processing part (18 of fig. 5) changes at least one selected from a 
position, a direction of a line of sight, and a focal length of the virtual point of view in 
accordance with a running state of the vehicle ( col. 21, line 41 -col. 22, line 6) note the image 
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processor (18) responds to the temporal and spatial patterns of infrared signals detected by 
image capture devices (cameras 14, 16) in order to determine (selected) the speed and distance 
and, thereby, the separation of the vehicles as well as the rate of change of separation of the 
vehicles as considered a running state of the vehicle). 

Re claim 2, Schofield further discloses wherein the image processing part (18 of fig. 5) 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view in accordance with a running speed of the vehicle (fig. 10 and fig. 11, 
note when the vehicle turns to the left or right, a position is changed so that the image capture 
devices (14 and 16 of fig. 5) picks up a new images based upon the changed position). 

Re claim 3, Schofield further discloses wherein the image processing part (18 of fig. 5) 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view, and controls capturing of an image outside a view range of the changed 
virtual point of view (col. 14, lines 1-29, see also fig. 7, note distances R, S, PI and P2). 

Re claim 4, Schofield further discloses wherein the image processing part (18 of fig. 5) 
controls the capturing of an image outside a view range of the changed virtual point of view by 
changing a model for image synthesis (an object). 

Re claim 5, Schofield further discloses wherein the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view in accordance with a steering angle of the vehicle (XVII of fig. 12, see also col. 17, lines 
31-36). 

Re claim 6, Schofield further discloses wherein the vehicle includes an object detection 
sensor (176 of fig. 21, note the object sensor 176 may be a distance-measuring device, such as 
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an active infrared sensor, an ultrasonic sensor, a radar sensor, or the like. Such object sensor is 
especially useful in determining the separation distance between the vehicle and objects in front 
of the vehicle. Preferably, object sensor 176 has a sensing field of view that is substantially 
coincident with the field of view of one or more of the image capture devices 14, 16) for 
detecting an obstacle, and the image processing part changes at least one selected from a 
position, a direction of a line of sight, and a focal length of the virtual point of view in 
accordance with results of detection by the object detecting sensor (176 of fig. 21). 

Re claim 7, Schofield further discloses wherein the image processing part (18 of fig. 5) 
includes an original mapping table (col. 14, lines 30-56) and generates a synthetic image using a 
mapping table that is cut out from the original mapping table, and the image processing part 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view by changing the mapping table to be cut out from the original mapping 
table (col. 14, lines 54-56). 

Re claim 8, Schofield further discloses an image processing apparatus comprising (fig. 
21) an image processing (18 of fig. 21) part for receiving images captured by a plurality of 
cameras (14 and 16 of fig. 21) shooting surroundings of a vehicle (10 of fig. 1) to generate a 
synthetic image viewed from a virtual point of view from these camera images, 

wherein the image processing part (18 of fig. 21) controls capturing of an image outside a 
view range of the virtual point of view in accordance with a running state of the vehicle (col. 14, 
lines 1-7). 

Re claim 9, Schofield further discloses a monitoring system comprising: a plurality of 
cameras (14 and 16 of fig. 21) shooting surroundings of a vehicle; an image processing part (18 
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of fig. 21) for receiving images captured by the plurality of cameras to generate a synthetic 
image viewed from a virtual point of view from these camera images; 

a display part (20 of fig. 7) for displaying the synthetic image, wherein the image 
processing part changes at least one selected from a position, a direction of a line of sight, and a 
focal length of the virtual point of view in accordance with a running state of the vehicle (100 
and 174 of fig. 21). 

Re claims 10, 16, Shimizu further discloses an image processing apparatus comprising an 
image processing part (18 of fig. 21) for receiving images captured by a plurality of cameras (14 
and 16 of fig. 21) shooting surroundings of a vehicle to generate a synthetic image from these 
camera images, 

wherein the image processing part (18 of fig. 21) generates an image including a first 
image (the captured image is from the camera 14 or 16) as the synthetic image, 

the second image being viewed (fig. 3, the right image being viewed on the display is 
different from the left image) from a viewpoint that is different from the virtual point of view of 
the first image in at least one selected from a position, a direction of a line of sight and a focal 
length, or the second image being different from the first image in a model (). 

Re claim 11, Schofield further discloses wherein the second image is at least one of the 
camera images (fig. 3). 

Re claims 17 and 23, Schofield further discloses an image processing apparatus 
comprising: 
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an image processing part (18 of fig. 2) for receiving images captured by a plurality of 
cameras (14 and 16 of fig. 7) shooting surroundings of a vehicle (10 of fig. 1) to generate a 
synthetic image from these camera images, wherein in the synthetic image, 

the image processing part (20 of fig. 5 and a display 20 of fig. 3) displays at least a part of 
a vehicle region (42 of fig. 3) where the vehicle is present, and an attention drawing region 
(Lanes, 70A, 70B of fig. 6) for drawing attention in which at least a part of the surroundings of 
the vehicle is shown. 

Re claim 18, Schofield discloses wherein the synthetic image is an image viewed from a 
virtual point of view that is set above the vehicle (fig. 3). 

Re claim 19, Schofield further discloses wherein the image processing part displays an 
illustration image or an actual image of the vehicle on the vehicle region (20 of fig. 3, see also 
col. 7, line 47-col. 8, line 12). 

Re claims 20-21, Schofield further discloses wherein the attention drawing region 
includes at least a part of a blind spot region around the vehicle that is not shot by any of the 
cameras (20 of fig. 3, note the display (20) is displaying the image that includes a part of blind 
spot. See also col. 9, lines 17-36). 

Re claim 22, Schofield further discloses wherein the image processing part determines a 
range of a region obtained by combining the blind spot region and the vehicle region, using 
region data showing a projection region of the vehicle in each camera image (76 of fig. 14; see 
also col. 9, lines 7-11). 

Re claim 24, Schofield discloses an image processing apparatus comprising an image 
processing part (18 of fig. 5) for receiving images captured by a plurality of cameras (14 and 16 
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of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these camera 
images, wherein the image processing part (18 of fig. 5) generates the synthetic image, using a 
mapping table (col. 14, lines 30-56) including first mapping data describing a correspondence 
relationship between pixels of the synthetic image and pixels of the camera images (Left image 
and Right image are from the outputs of the cameras 14 in figure 5, see also fig. 3), and second 
mapping data describing an identifier showing that a pixel of the synthetic image corresponds to 
pixel data other than the camera images (58 of fig. 3), and Schofield further discloses wherein 
the pixel data other than the camera images show the vehicle or a blind spot region that is present 
in at least a part of the surroundings of the vehicle (70A and 70B of fig. 6). 

Re claims 28 and 29, Schofield discloses an image processing apparatus comprising an 
image processing part (1 8 of fig. 5) for receiving images captured by a plurality of cameras (14 
and 16 of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these 
camera images, 

wherein the image processing part uses mapping data (col. 14, lines 30-56 ) describing a 
correspondence relationship between pixels of the synthetic image and a plurality of pixel data 
including one or both of pixel data of the camera images and pixel data other than the camera 
images, and describing a rate of necessity with respect to each of the pixel data, and weights each 
pixel data in accordance with the rate of necessity, thereby generating the pixel data of the pixels 
of the synthetic image (col. 14, line 54-col. 15, line 2). 
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Claim Rejections - 35 USC § 103 



4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 10-15 and 24-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schofield et al. (US 6,498,620 B2) in view of Shimizu (US 5,796,991). 

Re claims 10-15, and 24-27, Schofield teaches an image processing apparatus comprising 
an image processing part (18 of fig. 5) for receiving images captured by a plurality of cameras 
(14 and 16 of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these 
camera images, wherein the image processing part (18 of fig. 5) generates the synthetic image, 
using a mapping table (col. 14, lines 30-56) including first mapping data describing a 
correspondence relationship between pixels of the synthetic image and pixels of the camera 
images (Left image and Right image are from the outputs of the cameras 14 in figure 5, see also 
fig. 3), and second mapping data describing an identifier showing that a pixel of the synthetic 
image corresponds to pixel data other than the camera images (58 of fig. 3); wherein the pixel 
data other than the camera images show the vehicle or a blind spot region that is present in at 
least a part of the surroundings of the vehicle (70A and 70B of fig. 6). 

It is noted that Schofield does not teach the image processing part stores a predetermined 
image other than the camera images, and with respect to the pixel of the synthetic image, the 
second mapping data describe coordinate values corresponding to the pixel in the stored 
predetermined image; wherein the first image is a close view image showing the vehicle and 
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surroundings thereof, and the second image is a distant view image showing an area distant from 
the surrounding area of the vehicle that is shown by the close view image; wherein the image 
processing part arranges the distant view image around the close view image in the synthetic 
image; wherein the distant view image is an image having continuity with the close view image 
as claimed 

However, Shimizu further discloses wherein the image processing part stores a 
predetermined image other than the camera images (CG MODEL DATABASE, 244 of fig. 7), 
and with respect to the pixel of the synthetic image (213 of fig. 7), the second mapping data 
describe coordinate values corresponding to the pixel in the stored predetermined image (CG 
IMAGE FORMING UNIT, 235 of fig. 7); wherein the second image is at least one of the camera 
images (201L and 201R of fig. 7); wherein the first image is a close view image showing the 
vehicle and surroundings thereof, and the second image is a distant view image showing an area 
distant from the surrounding area of the vehicle that is shown by the close view image (fig. 6 A, 
fig. 10A-10E); wherein the image processing part arranges the distant view image around the 
close view image in the synthetic image (figs. 6A and 6B); wherein the distant view image is an 
image having continuity with the close view image (figs. 10A-10E); wherein the first image 
shows at least a part of the vehicle and at least a part of the surroundings of the vehicle, and the 
second image is obtained by enlarging at least a part of the region shown by the first image (figs. 
10A-10E). 

Taking the combined teachings of Schofield and Shimizu as a whole, it would have been 
obvious to one of ordinary skill in the art to incorporate the teachings of Shimizu into the image 
processing part (18 of fig. 5) of Schofield for the same purpose of synthesizing the images with 
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the predetermined model. Doing so would provide a natural appearance among images of virtual 
objects and an improved simulation environment. 

Conclusion 



Endo et al. (US 6,335,754 Bl) discloses synchronization between image data and 
location information for panoramic image synthesis. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung T. Vo whose telephone number is (703) 308-5874. The 
examiner can normally be reached on 6:30 AM - 3:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris. Kelley can be reached on (703) 305-4856. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



The prior art made of record and not relied upon is considered pertinent to applicants 



disclosure. 
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